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Abstract 

In this paper, we consider a model of impulsive recurrent neural networks with 
piecewise constant delay. The dynamics are presented by differential equations with 
discontinuities such as impulses at fixed moments and piecewise constant argument 
of generalized type. Sufficient conditions ensuring the existence, uniqueness and 
global asymptotic stability of the equilibrium point are obtained. By employing 
Green's function we derive new result of existence of the periodic solution. The 
global asymptotic stability of the solution is investigated. Examples with numerical 
simulations are given to validate the theoretical results. 
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1 Introduction 



Recurrent neural networks and impulsive recurrent neural networks have been 
investigated due to their extensive applications in classification of patterns, as- 
sociative memories, image processing, optimization problems, and other areas 
pTll20ll2T]|22]|23llMll25^^ It is well known that these appli- 

cations depend crucially on the dynamical behavior of the networks. For ex- 
ample, if a neural network is employed to solve some optimization problems, 
it is highly desirable for the neural network to have a unique globally stable 
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equilibrium |41f42f43f44f45f46f47f48] . Therefore, stability analysis of neural 

networks has received much attention and various stability conditions have 
been obtained over the past years. Another interesting subject is to study the 
dynamical behavior of existence of the periodic solutions in recurrent neural 
networks. These periodic solutions present periodic pattern and have been 
used in learning theory, which are meant to capture the idea that certain 
activities or motions are learned by repetition p3|T6] . 




In numerical simulations and practical implementations of neural networks, it 
is essential to formulate a discrete-time system, an analogue of the continuous- 
time system. Hence, stability for discrete-time neural networks has also re- 
ceived considerable attention from many researchers |35f36f37f38f39f40] . As 
we know, the reduction of differential equations with piecewise constant argu- 
ment to discrete equations has been the main and possibly a unique way of 
stability analysis for these equations [I^fSU] . Hence, one has not investigated 
the problem of stability completely, as only elements of a countable set were 
allowed to be discussed for initial moments. Finally, only equations which are 
linear with respect to the values of solutions at non-deviated moments of time 
have been investigated. That narrowed significantly the class of systems. In 
papers [SHHEin], the theory of differential equations with piecewise constant 
argument has been generalized by Akhmet. Later, Akhmet gathered all results 
for this type of differential equations in the book [2] . All of these equations are 
reduced to equivalent integral equations such that one can investigate many 
problems, which have not been solved properly by using discrete equations, i.e., 
existence and uniqueness of solutions, stability and existence of periodic solu- 
tions. Moreover, since we do not need additional assumptions on the reduced 
discrete equations, the new method requires more easily verifiable conditions, 
similar to those for ordinary differential equations. 

In this paper, we develop the model of recurrent neural networks to differ- 
ential equations with both impulses and piecewise constant argument of gen- 
eralized type. It is well known that impulsive differential equation [T|llfl2] 
is one of the basic instruments so the role of discontinuity has been under- 
stood better for the real world problems. In real world, many evolutionary 
processes are characterized by abrupt changes at certain time. These changes 
are called to be impulsive phenomena, which are included in many fields such 
as biology involving thresholds, bursting rhythm models, physics, chemistry, 
population dynamics, models in economics, optimal control, etc. In the litera- 
ture, recurrent neural networks have been developed by implementing impulses 
and piecewise constant delay pi^26|27|28f29f3U|31|32] issuing from different 
reasons: In implementation of electronic networks, the state of the networks 
is subject to instantaneous perturbations and experiences abrupt change at 
certain instants which may be caused by switching phenomenon, frequency 
change or other sudden noise. Furthermore, the dynamics of quasi-active den- 
drites with active spines is described by a system of point hot-spots (with 
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an integrate-and-fire process), see [331IM] for more details. This leads to the 
model of recurrent neural network with impulses. It is important to say that 
the neighbor moments of impulses may depend on each other. For example, 
the successive impulse moment may depend upon its predecessor. The rea- 
son for this phenomenon is the interior design of a neural network. On the 
other hand, due to the finite switching speed of amplifiers and transmission of 
signals in electronic networks or finite speed for signal propagation in neural 
networks, time delays exist [2T|22|24||25] . Moreover, the idea of involving de- 
layed arguments in the recurrent neural networks can be explained by the fact 
that we assume neural networks may "memorize" values of the phase variable 
at certain moments of time to utilize the values during middle process till the 
next moment. Thus, we arrive to differential equations with piecewise con- 
stant delay. Obviously, the distances between the "memorized" moments may 
be very variative. Consequently, the concept of generalized type of piecewise 
constant argument is fruitful for recurrent neural networks [7fS|5] . Therefore, 
it is possible to apply differential equations with both impulses and piecewise 
constant delay to neural networks theory. 

The intrinsic idea of the paper is that our model is not only from the appli- 
cations point of view, but also from a new system of differential equations. 
That is, we develop differential equations with piecewise constant argument 
of generalized type to a new class of systems; impulsive differential equations 
with piecewise constant delay and apply them to recurrent neural networks 
[3]mi5|6|8|9] . Another novelty of this paper is that the sequence of moments 
6k, G N, where the constancy of the argument changes, and the sequence of 
impulsive moments, r^, are different. More precisely, each moment r^, z G N, 
is an interior point of an interval {Okidk+i)- This gives to our investigations 
more biological sense, as well as provides new theoretical opportunities. 



2 Model description and preliminaries 

Let N = {0, 1,2,.. .} and IR+ = [0, oo) be the sets of natural and nonnegative 
real numbers, respectively, and denote a norm on by || ■ || where = 

m 

\ui\. Fix two real valued sequences 6 = {9k} , r = {vk} , /cGN, t H 9 = (j) 

i=l 

such that 9k < 9k+i with — )• oo as A; — )• oo and Tk < Tk+i with — )• oo 
as — )■ oo, and there exist two positive numbers 9,t such that 9k+i — 9k < 9 
and r < Tk+i — Tk, k E N. The condition of the empty intersection is caused 
by the investigation reasons. Otherwise, the proof of auxiliary results needs 
several additional assumptions. 

The main subject under investigation in this paper is the following impulsive 
recurrent neural networks with piecewise constant delay 
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x'iit) = -a,x,(t) + kjfj{xj{t)) (2.1a) 

m 

+ H Cij9jixj{(3{t))) + di, t^n 

Axi \t=rk=Iik{xi{T,:)), > 0, i = 1,2, ■ ■ ■ ,m, /c e N, (2.1b) 

where /3{t) = Ok (see Fig.[T]) if 9k < t < 9k+i, k eN, t E M+, is an identification 
function, Axi(rfc) denotes Xiijk) — Xi{T^), where Xiij^^) = hm/i_j.o- Xi{Tk + h). 
Moreover, m corresponds to the number of units in a neural network, Xi{t) 
stands for the state vector of the ith unit at time t, fj{xj(t)) and gj{xj{(3{t))) 
denote, respectively, the measures of activation to its incoming potentials of 
the unit j at time t and /3(t), bij,Cij,di are real constants, bij means the 
strength of the jth unit on the ith unit at time t, Cij infers the strength of the 
jth unit on the ith unit at time (3{t), di signifies the external bias on the ith 
unit and Oj represents the rate with which the ith unit will reset its potential 
to the resting state in isolation when it is disconnected from the network and 
external inputs. 




Fig. 1. The graph of the argument P{t). 

In the theory of differential equations with piecewise constant argument OHE], 
we take the function /3(t) = 6k if 9k ^ t < 9k+i, that is, P{t) is right contin- 
uous. However, as it is usually done in the theory of impulsive differential 
equations, at the points of discontinuity Tk of the solution, solutions are left 
continuous. Thus, the right continuity is more convenient assumption if one 
considers equations with piecewise constant arguments, and we shall assume 
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the continuity for both, impulsive moments and moments of the switching of 
constancy of the argument. 

We say that the function : M+ ^ is from the set PC^(M+, M™) if: 

(i) Lp is right continuous on M"*"; 

(ii) it is continuous everywhere except possibly moments r where it has dis- 
continuities of the first kind. 

Moreover, we introduce a set of functions PC^ue(IR''') 1^™") if we replace r by 
r U 6' in the last definition. In our paper, we understand that : — )■ M™ is 
a solution of ii cp e Pa(M+, M"^) and (p' G Paue(K+, M"). 

Throughout this paper, we assume the following hypotheses: 

(HI) there exist Lipschitz constants Lj,Lj > such that 

\Mu)-f,iv)\<Lf^\u-vl 

\9j{u) - 9j{v)\ <^\u-v\ 

for all u,v G M"", j = 1, 2, ■ ■ ■ , m; 
(H2) the impulsive operator /j^ : — )■ satisfies 

\Iik{u) - Iik{v)\ < i\u - v\ 

for all u,v & R™', i = 1, 2, ■ ■ ■ , m, /c G N where ^ is a positive Lipschitz 
constant. 

For the sake of convenience, we adopt the following notations in the sequel: 



ki = max ( a, 

l<i<m 



+ L{Y. I^iil j' ^2 = ^max i^Lf ^ \cji\j, h = max (^|4(0)| 



^4= max J2 \bji\\fii^)\ + Wji\\9iiO)\ 

l<i<m \ — ■ ' 
\?=1 



Denote by pk the number of points Xj in the interval (6*^, 6*^+1), k E N. We 
assume that p = max pi, < 00. 

Assume, additionally, that 



(H3) 



{ki + 2k2)d + 



< 1; 



(H4) k2e + [kiO + ep){l + k2e){l + if e''^'^ < 1. 
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2.1 Equilibrium for neural networks models with discontinuities 

In our paper, the main objects of discussion are the concepts of equihbria for 
discontinuous neural networks models. 

First of all, we must say that an equilibrium is a necessary attribute of any 
models for neural networks. Since, in the applications of Hopfield-type neural 
networks, when these networks can be considered as a nonlinear associative 
memory, the equilibrium states of the network serve as stored patterns and 
their stability means that the stored patterns can be retrieved representing 
the fundamental memories of the system P^14lll7fl8f26ll27f28f29f30f31] . To 
give more sense, the stable points of the phase space of the network are the 
fundamental memories, or prototype states of the network. For example, when 
a network has a pattern containing partial but sufficient information about 
one of the fundamental memories, we may represent it as a starting point in 
the phase space. Provided that the starting point is close to the stable point 
representing the memory being retrieved, finally, the system converges onto 
the memory state itself. Consequently, it can be said that Hopfield network is 
a dynamical system whose phase space contains a set of fixed (stable) points 
representing the fundamental memories of the system p^l4lll7p8] . This is 
obviously true for networks, which generalize the Hopfield one. For example. 
Recurrent neural networks, which are considered in our manuscript, Cohen- 
Grossberg neural Networks (Hopfield neural networks as a special version) [12] 
and Cellular neural networks [20] . 

Now, we want to give a general analysis of the equilibrium concept for discon- 
tinuous dynamics descibed by impulsive differential equations. Consider a x{t), 
motion of a neural network, assuming that it is discontinuous. Since the process 
is assumed right continuous, we have that x(rfc) — a;(r^) = /fc(x(r^)), k eN, 
where Ik{x) is an activation function. If one expects that a; = x* is an equilib- 
ria of the network, the necessarily Ik{x*) = 0, since x = x* is not a fixed point, 
otherwise. Thus, this condition is to be the cornerstone of our manuscript. One 
can see that the property of impulses to diminish at the equilibrium is not easy 
to derive from the mathematical analysis as it is true for continuous dynamics 
and it should be introduced in a special way |26f27f28f29f3Uf31] . In 
our paper we pay attention to the main subject more than usually. One must 
emphasise that despite zero impulses at equilibria, motions near eqiulibria ad- 
mit non-zero impulses, and it makes the systems considerable in the dynamics 
theory. Thus, if one recognize the zero right-hand side of a differential equa- 
tion at an equilibrium, then annihilated impulses have to be accepted, too. 
Mechanically, the zero impulses are the same as the zero velocity. If motionless 
points recognized, they must be accepted not only for smooth systems but also 
for discontinuous systems. To illustrate this point of view let us remember the 
impulsive differential equation in its general form 
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x' = f{t,x), t 7^ tk 
Ax \t=tk = Ik{x). 

The system can be written as 

oo 

x' = fit, x) + h{x)6(t - tk), 

k=l 

where 6{t) is the delta function. Then the equation for equihbria has the form 

oo 

fit,x*) + T.Ikix*)Sit-tk) = 0, (2.2) 

k=l 

which is very similar to that for ordinary differential equations. 

It is obvious that equilibria are easier for analysis if equations are autonomous, 
but they are also very popular for non-autonomous systems, in theory as well 
as in applications. Investigation of eqution fl2.2p is not more difficult, than 
that of f(t,x) = 0, in general. We suppose that theoretically proven existence 
of equilibria give us possibility to investigate concrete examples numerically 
and by simulations, admitting small errors, as it is done in our example with 
equation (15.151) in Section [51 

We denote the constant vector by * ) G M™', where the com- 

ponents X* are governed by the algebraic system 

m m 

= -a,x* + h,f,{x*) + c,,g,{x*)) + d^. (2.3) 
i=i i=i 

The proof of following lemma is very similar to that of Lemma 2.2 in [26] and 
therefore we omit it here. 

Lemma 2.1 Assume (HI) holds. If the condition 

m m 

tti > L{J2\bji\ + H l^i^l' i = l,---,m. 
i=i i=i 

is satisfied, then there exists a unique constant vector x* = (x*, . . . , x'^)'^ such 
that Ii2.3\) is valid. 

In this equilibrium point of equation (12.1bp . 
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Let us denote the set of all zeros of the impact activation functions by 
Q = i^x = {xi, . . . , x^f e I Iik{x) = 0, ^ = 1, . . . , m, /c G N} . 
From now on, we shall need the following assumption: 
(A) X* G VI. 



Theorem 2.1 If x* = {xl, . . . , xj^)^ is an equilibrium point of equation l[2.1h\) 
and the condition (A) is satisfied, then x* is an equilibrium point of \2.1\) . 



Now we need the following equivalence lemmas which will be used in the 
proof of next assertions. The proofs are omitted here, since it is similar in 



Lemma 2.2 A function x{t) = xit^to^x^) = {xi(t), ■ ■ ■ ,Xm(t))'^, where to is 
a fixed real number, is a solution of i\2.1\) on M"^ if and only if it is a solution, 
on M"*", of the following integral equation: 



to 



Y.bijfjixjis)) 



tO<Tk<t 



for i = 1, ■ ■ ■ ,m, t >to. 



Lemma 2.3 A function x{t) = x(t,to,x'^) = (xi(t),--- ,Xm(t))'^, where to is 
a fixed real number, is a solution of ^2. ij) on if and only if it is a solution, 
on M''', of the following integral equation: 



Xi(t)=X° + 



to 



-aiXi{s) + Yhijfj{xj{s)) 



+ ^(^ij9j{.Xj{(5{s))) + di 

for i = 1, ■ ■ ■ ,m, t > to- 
Consider the following system 



ds+ ^ik{Xi{Tk)), 

t0<Tk<t 



X'iit) = -aiXi{t) + Y Kfji^ji^)) + Y (^ij9j{Xj{(^r)) +di, t 7^ Tr 

j=i i=i (2.4) 

/S.Xi\t=r,= Iir{xi{T-)), i = 1 , 2, ■ ■ ■ , m, keN. 
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In the next lemma the conditions of existence and uniqueness of solutions are 
established for arbitrary initial moment ^. 



Lemma 2.4 Assume that conditions (HI) — (H3) are fulfilled, and fix r G N. 
Then for every {^,x^) G [6r,6r+i] x IR™ there exists a unique solution x{t) = 
x{t, ^, x°) = ■ ■ ■ , x^{t)f of ([g on [9r, 9r+i] with x(0 



Proof. Existence : Denote 'i9(t) = x{t,^,x^),i}{t) = , '(^^(t))^- From 

Lemma I2.3[ we have 



-ai^i{s) + hjfMj{s)) + CijQji^jier)) + di 



S.<Tr<t 



ds 
(2.5) 



Define a norm | \'d{t) | |o = max | |'i9()f:) 1 1 and construct the following sequences 

[dr, Sr + l] 

?9^(t), ?9°(t) = x°, i = 1, . . . , m, n > such that 



C^'(t) = ^° + 



i=i i=i 

i<Tr<t 



ds 



One can find that 



where 



K 



{ki + k2)e + lp\ \ \'d^\\+e(Ydi) +Omh + mpk 



i=l 



Since the condition (H3) implies {ki + k2)0 + £p < 1, then the sequences 
'&^{t) are convergent and their limits satisfy ( I2.5P on [6r, Or+i]. The existence 
is proved. 

Uniqueness : Let us denote the solutions of (12. ID by x^(t) = x(t, ^, x^), x^it) = 
x(t,^,x^), where Or < ^ < ^r+i- It is sufficient to check that for each t G 
[Or.er+il and = (xj,--- ,xlf,x^ = {x\,--- ,xlf G M™,x2 ^ xi, the 
condition x^{t) ^ x^{t). Then, we have 
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^ ( ft 

\x\t)-x\t)\\<\\x' -x^W+Y^l / 

i=l [-^^ 



+ Y^L!\c,,\\xKer)-xl{dr) 



ds + i i^ri^r )-xi[^r 

^<Tr<t 

rt 



<\\x^ - x^W + k2e\\x\er) - x\er)\\ + h j \\x\s) - x\s)\\ds 

E \\Arr)-x'{r-)\\. 

^<Tr<t 



Using Gronwall-Bellman Lemma for piecewise continuous functions p!T|T2] 
one can obtain that 



\x\t)-x\t)\\ < (^\\x^ - x^w + k2e\\x\er) - x^ier)\\yi + iy e 



p pkie 



Particularly, 



\x\er) - x\er)\\ < (^\\x^-x''\\ + he\\x\er)-x''{er)\\yi + ife''~'. 



Hence, 



\x\t)-x\t)\\ < 



(1 + £)Pe^i^ 
l-M(l + £)Pe'=i^ 



Ix'^ — x^l 



(2.6) 



Also, we peculiarly have 



\xHt-) - xHt-)\\ < 



(1 + £)Pe'=i^ 
1 - M(l + £)Pe*^i 



(2.7) 



On the other hand, assume on the contrary that there exists t G [6r, Or+i] such 
that x^{t) = x^{t). Then 
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m 



<h f' \\x\s) - X\s)\\ds + k2e\\x\er) - X\er)\\ 



^<Tr<t 



+£p\\x\t-) - x\rr 



(2. 



Consequently, substituting fl2.6p and (12. 7p in (12. Sp . we obtain 



\x'^ — x'^W < 



{ki + 2^2)^ + 



(2.9) 



Thus, one can see that (H3) contradicts with (12.91) . The lemma is proved. □ 

Theorem 2.2 Assume that conditions (HI) — (H3) are fulfilled. Then, for 
every {tQ,x^) G x M™, there exists a unique solution x{t) = x{t,to,x^) = 
{xi{t), ■ ■ ■ ,Xmit))'^, t > to, of h2. such that xito) = x^. 

Proof. Fix to ^ IR^- It is clear that there exists r G N such that to G [9r, Or+i)- 
Using previous lemma for = to, one can obtain that there exists a unique 
solution x(t) = x{t, to, x°) on [^, 6'r+i]. Next, we again apply the last lemma to 
obtain the unique solution on interval [Or+i, ^r+2)- The mathematical induction 
completes the proof. □ 



3 Global asymptotic stability 



In this section, we will focus our attention on giving sufficient conditions for 
the global asymptotic stability of the equilibrium, x* , of (12. ip based on lin- 
earization [5pT] . 

The system (12. ip can be simplified as follows. Substituting y{t) = x{t) — x* 
into (12. ip leads to 

m m 

y[{t) = -aiViit) + ^&*j0j(?/i(t)) + IZQjV^j(?/j(/3(t))), t ^ Tk 

3=1 0=1 (3.10) 

/\yi\t=r^=Wik{yi{T^)), i = l,2,---,m, keN, 
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where (/>,(%(t)) = f,{yj{t) + x*) - = gj{yj{t) + X*) - gj{x*) 

and WikiViir,, )) = likiViir,^ ) + x*) - Iik{x*) . From hypotheses (HI) and (H2), 
we have the following inequalities: |0j(-)| < < Lj\{-)\ and 

|w^..(-)l<^l(-)l- 



It is clear that the stability of the zero solution of f l3.10p is equivalent to the 
stability of the equilibrium x* of (12. ip . Therefore, in what follows, we discuss 
the stability of the zero solution of fl3.10p . 

First of all, we give the lemma below which is one of the most important results 
of the present paper. One can see that this lemma is generalized version of the 
lemmas in 



For simplicity of notation, we denote 



A = 1 - M + (he + ip){l + + iye 



-1 



Lemma 3.1 Let y{t) = {yi{t), • • • , ym{t)Y be a solution of Ii3.10\) and (HI) - (H4) 
be satisfied. Then, the following inequality 



\\ymm<M\ym (s.n) 

holds for all t G M+. 

Proof. Fix t G M^, there exists k E N such that t G [6*^,6*^+1). Then, from 
Lemma [2.31 we have 



\ym = T.\yM 

i=l 

m 

<ll2/(4)ll+E 



ai + Y,L{\bji\] \yi{s) 
3=1 ^ 

ds + i E \y^(^k 

tQ<'rk<t 



<(1 + 



k29)\\y{9,)\\ + h f \\y{s)\\ds + i E Mn 

•J 9k. 



tO<Tk<t 

Applying the analogue of Gronwall- Bellman Lemma [TT|T2] , we obtain 



\\y{t)\\<{l + he){l + ire''^~'\\y{0,) 
Particularly, 



(3.12) 
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||l/(r,-)|| < {l + he){l+lYe'~'\\y{e,)\\. (3.13) 
Moreover, for t G [6k, Ok+i), we also have 



\y{ek)\\<\\ym + k2e\\ym\ + h t Ms)\\ds 



k 



+^ E M'^k)\\- 

to<Tk<t 

The last inequality together with (I3.12p and f l3.13p imply 



Wkm<\\ym + 



\ym 



k^e + {kiO + ip)(i + k2e){i + if e'^i^ 

Thus, we have from condition (H4) that 

M0k)\\ < M\y{t)\l te[9k,9k+i). 

Therefore, (13.111) holds for all t G M^. This completes the proof of lemma. □ 

Now, we are ready to give sufficient conditions for the global asymptotic sta- 
bility of (12.11) . For convenience, we adopt the notation given below in the 
sequel: 



fi = max 

l<i<m 



n ^ 



From now on we need the following assumption: 
(H5) 7 — yU — \k2 — '•"'^^'^'•^ > 0, 7 = min a,. 

— l<i<m 

The next theorem is a modified version of the theorem in pjj , for our system. 



Theorem 3.1 Assume that (HI) — (H5) are fulfilled. Then, the zero solution 
of 113. 10\) is globally asymptotically stable. 

Proof. Let y(t) = (yi(t), ■ ■ ■ , ymit)V be an arbitrary solution of (I3.10p . From 
Lemma 12. 2[ we have 
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boll + E / 

i=l ["'^o 



y(t-s) 



<g-7(t-to)| 



t0<Tk<t 



yo\ \ + (/i + AA;2) / e 

J to 



^7(t-s)| 



|y(s)IMs 



-7(t-rfc)| 



2/(r.-)||. 



<0<T'fc<i 

Then, we can write the last inequahty as, 



Jto 



+^ E 



:,7(Tfc-t0) I 



t0<Tk<t 

By virtue of Gronwall-Bellman Lemma [H], we obtain 



where i{to,t) is the number of points in [to,^)- Then, we have 

||^^,)||<,-(7-.-A..-^)(.-.o)||^^||_ 



Hence, using the condition (H5), we see that the zero solution of system (13.101) 
is globally asymptotically stable. □ 



4 Existence of periodic solutions 



In this section, we shall discuss the existence of periodic solution of f|2.ip and 
its stability. To do so, we need the following assumptions: 

(H6) the sequences and 9k, k eN satisfy {uj,p) and (c(;,pi)-properties; that 
is, there are positive integers p and pi such that the equations r^+p = 
Tk + uj and 6k+pi = dk + ^ hold for all A; G N and I^-^-p = Ik for a fixed 
positive real period u. 

(H7) «i = 7^(^w (/i + Afca) + ip^ < 1, where 7^ = j^^. 

For Tfc and 9k, let [0,uj] n {r^lfceN = {n,--- ,Tp} and [0,w] H {6'fc}fcgN = 
{9i, ■ ■ ■ ,9p-^}, respectively. 
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Here, we will give the following version of the Poincare' criterion for system 
(12. ip which can be easily proved (see, also, [TT]). 



Lemma 4.1 Suppose that conditions (HI) — (H3) and (H6) are valid. Then, 
solution x{t) = x(t, to, = (xi, ■ ■ ■ , Xm)'^ of ^2. 1\) with x{to) = x° is 
u— periodic if and only if x{uj) = x{0). 

Theorem 4.1 Assume that conditions (HI) — (H3) and (H6) — (H7) are valid. 
Then system ^2.1\) has a unique u— periodic solution. 



Proof. To begin with, let us introduce a Banach space of periodic functions 

PC^ = {(p e PCrue(^+ I (p{t + u) = ip(t), t > 0} with the norm 
\Mo = max ||<^(t)||. 

0<t<uj 

Let ip{t) = {(pi(t), ■ ■ ■ ,ipmii)Y ^ PC^^ satisfying the inequality ||(y9(t)||o < h. 
Using Lemma 2.1, similarly to the proof in [IJJ, one can show that if G PC^^ 
then the system 



= -a-iXiit) + J2 hjfji^jit)) + Cij9j{^j{(3it))) + di, t^Tk, 
Axi \t=rk= liki'-Piir,:)), i = l,---,m, k = 1,2, - ■ ■ ,p 
has the unique u— periodic solution 



xKt)= rg.it, s 

Jo 

p 



Y hjfji^jis)) + Cijgj{iPj{l3{s))) + di 

3=1 j=l 



ds 



p 

1 

k=l 

where 



gi{t,s) = (1 - e 



which is known as Green's function [Tl] . Then, one can easily find that 

max |{^i(t, s)}i=i,...,m| = — . 

t,se[o,uj] 1 — e~'^''^ 

Define the operator : PCuj — )■ PC^j such that if 9? G PC^j, then 
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(^(^),(t)= rgi{t,s) 

Jo 

p 

+ ^gi{t,Tk)Iik{(Pi{Tk)), i = l,--- ,m 



ds 



k=l 



Now, we need to prove that maps PC^^ into itself. That is, we shall show 
that J^ip G PCuj for any ip e PC^j. It is easy to check that {J^(p){t) ~ 
{{J-'ip)i, ■ ■ ■ , {J-'(p)m)^ is cj— periodic function. Now, if 93 e PCt^, then 



ll-^^ll = E 



i=l 



ds 



k=l 
m -1 

<y — - — 

1=1 



j=l j=l 



i=i j=i 



ds + ^\Iik{iPi{Tk))\ 



k=l 



i=l 



3=1 j=l 



+ El^.^ll¥'.(o)l + Elc..lk.(o)l + c?. 



fe=l 



k=l 



^Ms)\\ + k^M(5{s) 



ds + lY.\\vi'^k)\ \ E^O +^mki + 

k=l ^ i=l 



In this periodical case, we take — max ( |7fe(0)| ) . Thus, it follows that 

i<fe<p \ / 



\\J^v\\o<T^\^[^^{fJ' + ^k2) + ip]\\(p\\o + uj[ E^O + i^'mk4, + mpk3 
< aih + a2- 



i=l 



Choose h such that a2 < h{l — ai), where a2 = T^^cu^^dj^ +(x!mk4+mpk^ . 
Then, Tip E PC^. '"^ 

Next, the proof is completed by showing that J-" is a contraction mapping. If 
(p'^,(p'^ e PCu,, then 
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\\:F^\t) - :Fv\t)\\ = Y.\{:F^'Ut) - {:Fv%{t) 



Hence, 



i=l 
m 



<E 



k=l 



<7^E 



+ E^f|c..ll^-(/3(^))-^-(/3(«))l 



k=l 



k=l 



ds 



WJ^ifi^ - Tip'^\\o<n{uj + \k2) + ip]\\^^ - ^'^Wo 



It follows from the condition (H7) that, J-" is a contraction mapping in PC^}- 
Consequently, by using Banach fixed point theorem, J-" has a unique fixed 
point ip* G PCij, such that J-'ip* = ip*. This completes the proof. □ 

Theorem 4.2 Assume that conditions (HI) — (H7) are valid. Then the peri- 
odic solution of Ii2. 1\) is globally asymptotically stable. 

Proof. By Theorem I4.H we know that (12. ip has an w— periodic solution 



z*{t) = {zl, ■ ■ ■ , z^y. Suppose that z{t) = {zi, 



is an arbitrary solu- 



, Zm — z^)'^ . Then, similar 



tionof dSD and let z{t) = z{t)-z*{t) = (zi-zl, 
to the proof of Theorem 13.11 one can show that it is globally asymptotically 
stable. 



Numerical simulations 



In this section, we give examples with numerical simulations to illustrate the 
theoretical results of the paper. In what follows, let 9k = k, Tk = {Ok + 
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6k+i)/2 = {2k + l)/2, k e'H be the sequence of the change of constancy for 
the argument and the sequence of impulsive action, respectively. 

Consider the following recurrent neural networks: 



dx(t) 
dt 



Ax{t) 



5 X 10"^ 

5 X 10-^ 
^2 X 10^2 3 X 10-3 
^3 X 10-3 5 X 10-3 
4(xi(r~)) 





10-4 2 X 10-A / tanh(^) 
10-4 3 X 10-4 ) \^tanh(^: 



tanh(2^; 



^2(^fc ) , 1 
40 2 , 







)-l 


;;) 






, k 


= 1, 



(5 



By simple calculation, one can see that the corresponding parameters in the 
conditions of Theorems 1221 EH gH O are ki_= 0.5001, fcs = 0.0046, L( = 
0.1, lI = 0.3, Ll = Ll = 0.2, I = 0.0250, 9 = t = 1, p = pi = l, j = 
0.5, A = 9.6421, /i = 0.00015, u = 1, TZ = 2.5415, ai = 0.1766. For these 
values, we can check that {H3) = 0.9032 < 1, (HA) = 0.8963 < 1, (//5) = 
0.4308 > and ai = 0.1766 < 1. So, it is easy to verify that fl5.14p satisfies 
the conditions of these theorems. Hence, the system of ( 15.14p has a 1-periodic 
solution which is globally asymptotically stable. Specifically, the simulation 
results with some initial points are shown in Fig. 1. and Fig. 2. We deduce 
that the non-smoothness at 6'^, A; G N is not seen by numerical simulations due 
to the choosing the parameters small enough to satisfy the theorems. Hence, 
the smallness hides the non-smoothness. 
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01 23456789 

t 




t 



Fig. 2. Transient behavior of the recurrent neural networks for the system ()5.14p 
with the initial points [0,0]"^ and [7,7]^. 




Fig. 3. Eventually 1-periodic solutions of system (I5.14p with the initial points [0, 0]^ 
and [7,7]^. 
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On the other hand, in the following example, we illustrate a globally stable 
equilibrium appearance for our system of differential equations: 



^ ^ _ I 5 X 10-1 \ / x,{t) \ ^ 1 10-4 2 X 10-3 \ / tanh(2^) 
' 5x10-1/ \x2{t)) \W-^ 3x10-3 ) \^tanh(22:^) 
2 X 10-2 3 X 10-2 

(5. 



Ax{t) 



3 X 10-2 5 X 10-2 










) 







where x* = 2.0987, = 2.1577. One can check that the point x* = {xl,X2) 
satisfies the algebraic system 



2 2 

- aiX* + ^ hjfj{x*) + J2 Cij9jix*)) + di = 0, 



(5.16) 



approximately. And it is clear that I{x*) = for i = 1,2. By simple calcula- 
tion, we can see that all conditions of Theorem 12.11 are satisfied and the point 
X* is a solution of f l5.16p . approximately with the error, which is less than 
10-11 (evaluated by MATLAB). 

The simulation, where the initial value is chosen as [10, 10]"^ , is shown in Fig. 
3 and it illustrates that all trajectories converge to x*. 




Fig. 4. The first and the second coordinates of the solution for the the system (I5.15P 
with the initial point [10, 10]"^ approaches and X2, respectively, as time increases. 

Now, let us take the parameters so that the non-smoothness can also be seen. 
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Consider the following recurrent neural networks with non-smooth and impact 
activations: 



dx{t) 
(it 



Ax{t) 





5 1 



tanh(xi(/3(t)))\ / 1 
tanh(x2(/3(t))) / \ 1 



10 3\ / tanh(xi(t))\ 
I tanh(x2(t)) J 



3 



+ 



Tfc, k= 1,2, 



(5.17) 



Clearly, one can see that our parameters are big now. Therefore, the system 
of equations (15.171) does not satisfy the conditions of the theorems. However, 
we can see the non-smoothness of the solution with the initial value [0, 0]"^, 
which is illustrated by simulations in Fig.3. and Fig. 4. 




0.5 1 



Fig. 5. The impact and non-smoothness are seen at discontinuity points 
Tfc (0.5; 1.5; 2.5; 3.5) and at switching points 0^ (1;2;3), respectively. 
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t 

Fig. 6. Eventually 1-periodic solutions of system (j5.17p with the initial point [0, 0]"^. 
6 Conclusions 

This is the first time that global asymptotic stability of periodic solutions for 
recurrent neural networks with both impulses and piecewise constant delay is 
considered. Furthermore, our model gives new ideas not only from the imple- 
mentation point of view, but also from the system of differential equations. In 
other words, we develop differential equations with piecewise constant argu- 
ment to a new class of system, so called impulsive differential equations with 
piecewise constant delay. For applications, we have also nice properties on the 
system of equations that the moments of discontinuity and switching mo- 
ments of constancy of arguments 6k are not related to each other. That is, our 
investigations are more applicable to the real world problems like recurrent 
neural networks. Finally, the results given in this paper could be developed 
for more complex systems |10j . 



References 

[1] M. Akhmet, Principles of Discontinuous Dynamical Systems, Springer, New 
York, 2010. 

[2] M. Akhmet, Nonlinear Hybrid Continuous/Discrete Time Models, Atlantis 
Press, Amsterdam-Paris, 2011. 



22 



[3] M. U. Akhmct, On the integral manifolds of the differential equations with 
piecewise constant argument of generalized type, Proceedings of the Conference 
on Differential and Difference Equations at the Florida Institute of Technology, 
August 1-5, 2005, Melbourne, Florida, Editors: R.P. Agarwal and K. Perera, 
Hindawi Publishing Corporation, (2006) 11-20. 

[4] M. U. Akhmet, On the reduction principle for differential equations with 
piecewise constant argument of generalized type, J. Math. Anal. Appl. 336 
(2007) 646-663. 

[5] M. U. Akhmct, Stability of differential equations with piecewise constant 
arguments of generalized type. Nonlinear Analysis 68 (2008) 794-803. 

[6] M. U. Akhmet and D. Arugaslan, Lyapunov-Razumikhin method for differential 
equations with piecewise constant argument. Discrete Cont. Dyn. S., 25:2 (2009) 
457-466. 

[7] M. U. Akhmet, D. Arugaslan and E. Yilmaz, Stability in cellular neural 
networks with a piecewise constant argument, J. Comput. Appl. Math., 233, 
2010 2365-2373. 

[8] M. U. Akhmet and E. Yilmaz, Impulsive Hopfield-type neural networks system 
with piecewise constant argumet. Nonlinear Anal: Real World Applications, 11 
(2010) 2584-2593. 

[9] M. U. Akhmet, D. Arugaslan and E. Yilmaz, Stability analysis of recurrent 
neural networks with piecewise constant argument of generalized type, Neural 
Networks, 23 (2010) 805-811. 

[10] M. U. Akhmet, Dynamical synthesis of quasi-minimal sets, International 
Journal of Bifurcation and Chaos, Vol. 19, No. 7 (2009), 1-5. 

[11] A. M. Samoilenko and N.A. Perestyuk, Impulsive Differential Equations, World 

Scientifc, Singapore, 1995. 

[12] V. Lakshmikantham, D.D. Bainov and P.S. Simeonov, Theory of Impulsive 
Differential Equations, in: Series in Modern Applied Mathematics, vol. 6, World 
Scientific, Singapore, 1989. 

[13] S. Haykin, Neural Networks: A comprehensive Foundations, Second edition, 
Tsinghua Press, Beijing, 2001. 

[14] J. F. Kolen and S. C. Kremer, A Field Guide to Dynamical Recurrent Networks, 
IEEE Press, New York, 2001. 

[15] H. Huang, D. W. C. Hob and J. Cao, Analysis of global exponential stability and 
periodic solutions of neural networks with time- varying delays. Neural Networks 
18 (2005) 161-170. 

[16] S. Townlcy, A. Ilchmann, M. G. Weib, W. Mcclements, A. C. Ruiz, D. H. Owens 
and D. Pratzel-Wolters, Existence and learning of oscillations in recurrent 
neural networks, IEEE Transactions on Neural Networks, 11 (1) (2000) 205- 
214. 



23 



[17] J. J. Hopfield, Neurons with graded response have collective computational 
properties like those of two-stage neurons, Proc. Nat. Acad. Sci. Biol. 81 (1984) 
3088-3092. 

[18] J. J. Hopfield, Neural networks and physical systems with emergent collective 
computational abilities, Proc. Nat. Acad. Sci. Biol. 71 (1982) 2554-2558. 

[19] M. A. Cohen and S. Grossberg, Absolute stability of global pattern 
formation and parallel memory storage by competitive neural networks, IEEE 
Transactions SMC-13, pp. 815-826, 1983. 

[20] L. O. Chua and L. Yang, Cellular neural networks: Theory, IEEE Trans. Circuits 
Syst. 35 (1988) 1257-1272. 

[21] L. O. Chua and L. Yang, Cellular neural networks: Applications, IEEE Trans. 
Circuits Syst. 35 (1988) 1273-1290. 

[22] L. O. Chua and T. Roska, Cellular neural networks with nonlinear and delay 
type template elements and non-uniform grids. International Journal of Circuit 
Theory and Applications 20 (1992) 449-451. 

[23] A. N. Michel, J. A. Farrcll and W. Porod, Qualitative analysis of neural 
networks, IEEE Trans. Circuits Systems 36 (1989) 229-243. 

[24] P. P. Civalleri, M. Gilli and L. Pandolfi, On stability of cellular neural networks 
with delay, IEEE Trans. Circuits Syst. I (40) (1993) 157-164. 

[25] S. Coombcs and C. Laing, Delays in activity-based neural networks, Phil. Trans. 
R. Soc. A 2009 367, 1117-1129. 

[26] K. Gopalsamy, Stability of artificial neural networks with impulses, Appl. Math. 
Comput., 154 (2004) 783-813. 

[27] S. Mohammad, Exponential stability in Hopfield-type neural networks with 
impulses. Chaos Solitons and Fractals 32 (2007) 456-467. 

[28] Z. H. Guan, J. Lam and G. Chen , On impulsive autoassociative neural 
networks. Neural Networks 13 (2000) 63-9. 

[29] Z. H. Guan and G. Chen , On delayed impulsive Hopfield neural networks. 
Neural Networks 12 (1999) 273-280. 

[30] D. Xu and Z. Yang, Impulsive delay differential inequality and stability of neural 
networks, J. Math. Anal. Appl. 305 (2005) 107-120. 

[31] Y. Zhang and J. Sun, Stability of impulsive neural networks with time delays, 
Phys. Lett. A 348 (2005) 44-50. 

[32] Y. Yang and J. Cao, Stability and periodicity in delayed cclhilar neural networks 
with impulsive effects. Nonlinear Analysis: Real World Applications 8 (2007) 
362-374. 

[33] Y. Timofeeva, Travelling waves in a model of quasi-active dendrites with active 
spines, Physica D 239 (2010) 494-503. 



24 



[34] S. Coombes and P.C. Bressloff, Solitary waves in a model of dendritic cable 
with active spines, SIAM J. Appl. Math. 61 (2) (2000) 432-453. 

[35] Y. Liu, Z. Wang and X. Liu, Asymptotic stability for neural networks with 
mixed time-delays: The discrete-time case. Neural Networks 22 (2009) 67-74. 

[36] Y. Liu , Z. Wang , A. Serrano and X. Liu , Discrete-time recurrent neural 
networks with time-varying delays: Exponential stability analysis. Physics 
Letters A 362 (2007) 480-488. 

[37] N. E. Barabanov and D. V. Prokhorov, Stability analysis of discrete-time 
recurrent neural networks. IEEE Trans. Neural Networks 13(2) (2002) 292-303. 

[38] J. Liang, J. Cao and J. Lam, Convergence of discrete-time recurrent neural 
networks with variable delay, Internat. J. Bifur. Chaos 15 (2005) 581-595. 

[39] X. Zhao, Global exponential stability of discrete-time recurrent neural networks 
with impulses. Nonlinear Analysis 71 (2009) e2873-e2878. 

[40] S. Mohamad, Global exponential stability in continuous-time and discrete-time 
delayed bidirectional neural networks, Physica D 159 (2001) 233-251. 

[41] E. Yucel and S. Arik, New exponential stability results for delayed neural 
networks with time varying delays, Physica D, 191 (2004) 314-322. 

[42] J. Cao, Global stability analysis in delayed cellular neural networks. Physical 
Review E, 59, (1999) 5940-5944. 

[43] J. D. Cao and D. M. Zhou, Stability analysis of delayed cellular neural networks. 
Neural Networks, 11, (1998) 16011605. 

[44] X. M. Li, L. H. Huang and H. Y. Zhu, Global stability of cellular neural networks 
with constant and variable delays. Nonlinear Analysis, 53, (2003) 319-333. 

[45] S. Xu , Y. Chu and J. Lu, New results on global exponential stability of recurrent 
neural networks with time-varying delays, Phys. Lett. A 352 (2006) 371-379. 

[46] H. Huang, J. Cao and J. Wang, Global exponential stability and periodic 
solutions of recurrent neural networks with delays, Phys. Lett. A 298 (2002) 
393-404. 

[47] P. V. D. Driessche and X. Zou, Global attractivity in delayed Hopfield neural 
network models, SIAM J. Appl. Math. Vol. 58 No. 6 (1998) 1878-1890. 

[48] Z. G. Zeng and J. Wang, Improved conditions for global exponential stability 
of recurrent neural networks with time-varying delays, IEEE Trans. Neural 
Networks 17 (3) (2006) 623-635. 

[49] K. L. Cooke and J. Wiener, Retarded differential equations with piecewise 
constant delays, J. Math. Anal. Appl. 99 (1984) 265-297. 

[50] J. Wiener, Generalized Solutions of Functional Differential Equations, World 
Scientific, Singapore, 1993. 



25 



